Discovering Conditional Functional Dependencies to Detect Data Inconsistencies

Poor quality data is a growing and costly problem that affects many enterprises across all aspects of their business ranging from operational efficiency to revenue protection. In this paper, we present an approach that efficiently and robustly discovers conditional functional dependencies for detecting inconsistencies in data and hence improves data quality. We evaluate our approach empirically on three real-world data sets, and show that our approach performs well on these sets across several dimensions such as precision, recall, and runtime. We also compare our approach to an established solution and show that our approach outperforms this solution across the same dimensions. Finally, we describe efforts to deploy our approach as part of an enterprise tool being developed at Accenture to accelerate data quality efforts such as data profiling and cleansing.